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Abstract 

South Africa participated in the Third International Mathematics and Science Study (TIMSS) in 1995 and 
its repeat in 1999^00) under the auspices of the International Association for the evaluation of 
Educational Achievement (IEA). However, in 1995 none of the data on school or teacher level could be 
analysed to provide the context for the students' poor achievements in mathematics and science in 1995. 
With the 1999 data now available at both school and teacher level in addition to the student level data, 
this backdrop to the results can be provided. Firstly the results of the student’s performance in 
mathematics and English is given. Thereafter, questions on school level regarding the leadership of the 
school, the physical conditions within the schools, the students' behaviour, and the schools' expectations 
of parents were investigated in relation to mathematics achievement. 

Path analysis (using Partial Least Square analysis) was conducted on the TIMSS-R data to explore the 
effect of language and other factors at school level within South African schools on the students' 
performance in mathematics. The work is currently in progress, however the initial results already 
indicate the strong relationship between English language proficiency and South African pupils’ 
achievement in mathematics. The first results from the school level analysis reveals a relationship 
between the location of the schools, the number of first language speakers in the class and the role of 
teacher unions influencing the curriculum, to pupils’ achievement in mathematics. These and other 
results of this research are discussed in the paper. 



3 



1 Although South Africa actually collected its data in 1998 along with the Southern hemisphere countries, the study is referred to as TIMSS 1999 
internationally and therefore this convention has been applied in this paper. 
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1. Introduction 

At the end of the 20 th century, it was estimated that about 140 million people in sub-Saharan 
Africa could not read or write. Amongst South Africa's multi-cultural society of 39 million people, 
more than 3 million adults over the age of 16 have never attended school and more than 2.5 
million attended only a few years in school and are functionally illiterate. Much of this is directly 
attributable to the decades of Apartheid policies implemented under the nationalist government 
of South Africa. These separatist policies forced cultural groups apart under the guise of 
separate development. The education system became divided with children of each race group 
attending school separated on the basis of these racial groupings. Schools for white children 
received much more funding than others, had better facilities, were better equipped and had 
better qualified teachers. Therefore, in addition to the other challenges facing the rest of Africa, 
South Africa has a set of special circumstances to deal with. 

One of the most noticeable changes after the democratic elections in 1994, was the dramatic 
change 2 in the demography of students at previously white, coloured and Indian schools. 
Students from the townships (speaking mostly African-languages at home) poured into all of 
these schools, whilst many Indian and coloured (of mixed-race) children moved into former 
white government schools. In areas like Johannesburg where the diversity of languages is vast, 
there are obvious consequences for instruction in the classrooms. One of the key problems is 
that in the majority of schools the official language of instruction and the mother tongue of the 
teachers and/or the students are different. It is believed that as a result students' achievement 
in mathematics (and other subjects) is negatively affected. Research evidence (Joint Education 
Trust, 1998, Colvyn and Lebethe, 1998, Clarkson, 1991, Ellerton, 1991, Garden and 
Livingstone, 1989) suggests that language plays an important role in pupils’ learning and that 
pupils learning in a second language may be disadvantaged. It is hypothesised therefore that 
pupils performing poorly in the English language test will also perform poorly in the TIMSS 
mathematics test. 

This paper covers only part of a larger research project (Howie, 2000), that analyses the 
performance of the South African students in mathematics and science within an international 
comparative perspective as participant in the Third International Mathematics and Science 
Study-Repeat (TIMSS-R). TIMSS-R was conducted in 1998/1999 (and TIMSS in 1995 (see 
Howie, 1996)) under the auspices of the International Association for the Evaluation of 
Educational Achievement (IEA). 

This study builds on the work done by a number of researchers from other countries 
(Mohandos, 1999 and Afrassa, 1998 amongst others) and most recently upon the study 
conducted by Bos (2000) on student achievement in mathematics in the Netherlands. 
Ultimately the project aims to describe and explore the performance of the South African 
students in mathematics and the relationship between mathematics achievement and students’ 
proficiency in English, as well as other background variables. The factors relating to the 
students performance in mathematics and English language proficiency will be explored in 
relation to the background information that was collected from the students, teachers and 
principals of the schools included in the study. 

The aim of the exploration is also to try to ascertain which of the two levels (student-level or 
class/school-level) has the most influence on students’ achievement mathematics in the context 



2 The integration of different race groups in former white, Indian and Coloured schools had been taking place since the abolition of the group 
areas act in 1990 and in private schools since about 1976, however it was after 1994, that this increased rapidly. 
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of South Africa. Evidence from previous research suggests that whilst student-level factors 
have more influence on students’ achievement in developed countries (Husen, 1967), school- 
level factors play a greater role with regard to students’ achievement in developing countries 
(Heyneman, 1975 and Keeves, 1994). It is therefore believed that school-level factors are likely 
to play a more significant role with regard to South African Grade 8 students’ achievement in 
mathematics than student-level factors. However, it is also recognised that due to the disparity 
in the home backgrounds of the students that the home factors and school factors may play a 
greater or lesser role depending on the background of different groups of students. Therefore, 
the variables relating to the home background will be explored to determine how or whether 
they affect the achievement of different groups of students differently. In addition, the school 
factors will also be investigated to see how or whether they affect the achievement of different 
groups of students in divergent ways. 

This paper covers only the initial part of this exploration of the TIMSS-R data involving 
multivariate and multilevel analysis. Here the results of the English language test are given in 
addition to the overall TIMSS-R mathematics results. Furthermore, attempts are made to find 
factors at school-level that helps to explain the variance within the South Africa student’s 
achievement data. Factors relating to the leadership, the physical conditions, the school's 
expectations of the parents and the school environment are explored here in relation to the 
performance of South African students in TIMSS-R. 

The rest of the paper comprises the following: a summary of the development of language 
policy and practice in education in South African is given in Section 2. Section 3 provides a 
brief overview of some of the literature dealing with school level factors and their relationship to 
achievement. Thereafter, a short description of the South African education context is given in 
section 4. Section 5 provides the reader with some background information on the TIMSS-R 
project. The research design is described in section 6 and includes the conceptual framework 
for the larger research project. Finally section 7 contains some of the results of the study and 
preliminary conclusions about the results to date. 



2. Language policy and practice in South African schools 

Like so much of South Africa’s history, different policies were applied to different race groups. 
The policy in previously Coloured, Indian and White schools has stayed much the same since 
1910 with English and Afrikaans (previously Dutch) being the official languages of the Union of 
South Africa. This meant that either English or Afrikaans was used as the medium of instruction 
with the second language being taught as a compulsory subject for all students. However, in 
1948 when the nationalist party came to power, the policy changed and English and Afrikaans- 
speaking children were separated and sent to different schools where their mother tongue 
automatically became the language of instruction. Schools for Indians generally followed the 
English-speaking schools examples using English as the medium of instruction and offering 
Afrikaans as a compulsory subject. Coloured schools varied with some opting for English and 
others for Afrikaans as the medium of instruction and sometimes within one school both 
languages were used for instruction with students streamed for this purpose. 

In African schools initially, either English or Afrikaans were used as the medium of instruction, 
with the majority of schools opting for English. However, the Nationalist Party’s rise to power 
changed the policy and practice of language in education. Mother tongue was to be used for the 
first eight years of schooling (the whole of primary schooling) and English and Afrikaans 
became compulsory subjects from the first year of schooling. This change became the focus of 
the opposition to Bantu Education. With the introduction of and separate development of 
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independent homelands and self-governing territories, the language policies evolved differently 
to the then government's expectations. These opted for mother tongue instruction in only the 
first four years of schooling from the 1960's and by 1974 all had decided (with one exception) on 
using only English as the official language of instruction from year 5 onwards. During the 1980’s 
different options became available to parents of students at African schools as pupils could 
either start immediately with the medium of instruction to be used in the long term, or, have a 
sudden transfer from the mother tongue to a second language medium or, thirdly, a graduated 
transfer from the mother tongue to a second language medium. It would appear that the 
majority of schools adopted the first option of English as the language of instruction from Grade 
1, although no audit has been conducted to substantiate the anecdotal evidence available. 

The 1990's saw a multitude of policy documents on language-in-education. Although there was 
widespread consultation, there was also increased debate on bilingualism and multilingualism. 
In general, the policy documents promote the concepts of multilingualism. The importance of all 
South Africa's languages is stressed and at school level, the documents promote bilingualism at 
the very least but prefer that learners should learn at least three languages. The country's 
constitution asserts that the importance of " creating conditions for the development and 
promotion of the equal use of all official South African languages ". The White Paper on 
Education and Training (1995) states that the "language in education policy must accommodate 
the right to be instructed in a language chosen by the learner, where this is reasonably 
practical". However neither the White Paper or the Department of Education's Language in 
Education Policy (1997) places any pressure on the schools to offer particular languages. They 
only encourage schools, which are "willing and able to offer more than one language medium in 
order to accommodate parental or learners' preferences". Ultimately the decision on the 
languages of instruction and those offered as subjects is left to the school governing bodies. 
The Schools Act (1996) stipulates that these bodies are required to "announce the schools' 
language policy and to state how it will promote multi-lingualism through a variety of measures. 

The difficulty of this is that schools find themselves in a quandry as to how to introduce such 
changes with cuts in the education budgets, frozen posts and redeployment of teachers have 
left little space for schools to enter into new course directions. The absence of concrete 
implementation strategy from the government means in that in all likelihood, past practices will 
continue and the goal of a multi-lingual society will disappear. 

One of the most noticeable changes after 1994, was the dramatic change in the demography of 
students at previously white, coloured and Indian schools. African-language speaking students 
from the townships poured into all of these schools, whilst many Indian and coloured children 
moved into former white government schools. In areas like Johannesburg where the diversity 
of languages is vast, there are obvious consequences for instruction in the classrooms. One of 
the results of urbanisation, is that many students do not have an obvious primary language nor 
have the desire to identify with any one linguistic group (Pirie, 1984 and Brown, 1998). Instead 
many students speak a number of languages and in some townships like Soweto, develop a 
dialect made up of words from a variety of African languages, English and Afrikaans. However, 
it is also true that they may not be fluent in all of these languages or even truly fluent in even 
one language. 

The multicultural communities created by the dissolving of Apartheid and increased mobility of 
families seeking upward mobility means that issues around language in education are complex. 
Effective instruction is dependent largely on good communication under-pined by competence 
by both teachers and students in the language of instruction. However, the reality is that this is 
a problem in the majority of schools where the language of instruction used and the mother 
tongue of the teachers and/or the students is different. 
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3. School level factors related to achievement 

Research focusing on schools has been underway for more than 30 years. During this time, 
research has assumed a number of mantles namely school effectiveness, school improvement, 
school reform, school development, and school restructuring amongst others. Through this 
research a number of factors related to achievement have been uncovered at different levels in 
the education system, i.e. at school-, class- and student-level. Many authors attribute the start 
of this type of research to the team under Coleman (1966) who found that home background in 
the United States of America predicted by far the greatest variance in achievement outcomes. 
Using multiple regression analysis, Coleman reported that poverty and class were responsible 
for predicting achievement more reliably than school factors. School-level factors have 
traditionally explained a low percentage of variance in many research projects primarily 
conducted in developed contexts. Reynolds and Cuttance (1992) reviewed a number of studies 
and found only 8-1 5% of variance attributable to school factors. Most of the research done so 
far has been in the developed nations and largely in Western Europe and the United States of 
America. 

A number of studies also tried to prove the effectiveness of schools. New techniques were 
developed to do this. Edmond’s 5-factor model (1979) was developed through a longitudinal 
study and Rutter et al (1979) and Mortimore et al (1988) used complex data and multilevel 
analysis techniques for the first time. 

A variety of factors have been found that influence achievement. Specifically, it has been 
reported, in a review by Greenwald, Hedges and Laine (1996) that a number of studies found 
class size to have a minor effect on achievement. Fuller and Clarke (1994) identified three 
factors - textbooks, teacher quality and time as being key factors emerging from school 
effectiveness research. These are also mentioned by Riddell (1997) and in a review by 
Creemers (1996). Leadership, organisation and management are identified as important factors 
by school effectiveness researchers, whilst school improvement researchers have concentrated 
on decision-making, within-school hierarchy and communication. Recent findings in school 
effectiveness studies show that school-level factors influence achievement far less than do 
factors at the class-level. However, as this research is largely based in developed countries, 
the question is whether this is also the case in less-developed nations. 

In the past few years, calls have been made to link school interventions and contextual 
information with student achievement data. West and Hopkins (1996) stated that a school 
improvement strategy needs to be based on data about student performance claiming that 
different achievement profiles require different kinds of intervention. 

As mentioned earlier, the situation in developed and developing countries may well be different 
in relation to outcomes in research on school level and in which factors influence student 
achievement. One important difference is that resources at schools are important in developing 
countries. The World Bank (1995) lists libraries, time on task, homework, textbook provision, 
teacher knowledge, teacher experience, laboratories, teacher salaries and class size as 
important for effective schooling in developing countries. Other influential factors found are 
teacher expertise and competence, strong leadership, clear organisation of the school day and 
the learning programme (time and opportunity) and community and parental involvement in 
school governance (Muller, 2000:8). In South Africa, an investigation into well performing 
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schools by Crouch and Mabogoane, (1998) found that only 25% of achievement was 
explainable by resource availability. The variation in findings illustrates the importance of the 
context of the research. 

Not only is there a difference between developed and developing countries but there is also a 
significant difference between the variance in achievement explained at different education 
levels between and within-countries. Reynolds (1998:1279) claims that classroom-level has 
“maybe two or three times the influence on student achievement than the school level does". 

Scheerens (1998) conducted an extensive review of the literature on school effectiveness and 
the synthesis of the research outcomes revealed support for Reynolds earlier conclusion, 
namely that factors at classroom level correlate generally more highly with achievement than 
those at school level. 

In another review of literature at school level Muller (2000:17) concludes that home background 
is more influential than school “ because most of the damage is done before the children go to 
school". However, the complexity and peculiarities of schools maybe magnified by highly 
disadvantaged settings such as those in South Africa and other developing countries. There is a 
need to explore and disentangle the multiple associations and divergent outcomes derived from 
the same set of input variables. Multi-level modelling may be seen as an aid to do this, hence 
the growing tendency by researchers to use this technique. It is clear that although researchers 
may be approaching data from different perspectives (i.e. school improvement or school 
effectiveness), there is considerable interest in ascertaining the reasons related to the 
successful learning of students in schools across the world. Seemingly instructional factors at 
classroom level are more important than factors at school-level. In the developed world, 
researchers maintain that home background factors predict achievement of children in those 
countries. It is with these results in mind that this research was undertaken, to explore and 
investigate to what extent factors at school level had any influence on the achievement of South 
African students in mathematics. 

4. Education in South Africa 

Education in South Africa is compulsory and “free 3 ” for grades 1 to 9, and non-compulsory for 
grades 10 to 12. Students are only expected to pay fees for grades 10 to 12, but educational 
user fees are widespread in all the grades. Primary school spans grades 1 to 7, whilst grades 8 
to 12 constitutes the secondary school. In 1996, there were 12 million learners enroled at 
schools in South Africa, of whom two thirds were in primary school. 

The majority of South African secondary schools are comprehensive; however, there are a 
limited number of schools that provide commercial or technical subjects. The schools are either 
funded entirely by the state, or funded partly by the state (state-aided) or finally are funded 
privately. In 1996 all provinces had teacher/learner ratios below official targets of 40:1 for 
primary schools, and 35:1 for secondary schools. In general, many teachers in government 
schools, especially where there are non fee-paying parents, are faced with large classes. Only a 
small proportion of all learners (3%) in South Africa is enroled at private schools. The average 
teacher/learner ratio at 25: 1 in these schools is more favourable than that in government funded 
schools; and it can even be as low as 14:1. 

A considerable number of schools in South Africa suffer serious shortcomings, ranging from 
poor access to water, telephones and electricity, to the poor condition of many school buildings. 



3 However, schools are entitled and do ask parents to pay schools fees, but pupils whose parents claim that they are unable to pay 
may still attend the school and are constitutionally entided to do so. 
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In an HSRC survey of school needs (1996), 12% of schools had no sanitation facilities, 24% 
had no water within walking distance, and 59% had no telephones. Only 49% of schools had 
adequate provision of textbooks. Many teachers are also not qualified to teach their subjects. 
Although 85% of mathematics teachers are professionally qualified, only 50% have specialised 
in mathematics (Arnott et al, 1996) and sometimes for no more than one year post school 
equivalent. These conditions apply especially in former Department of Education and training 
Schools and the former homelands, both of which catered for African pupils only. 

In 1993, of the 157 701 pupils who wrote the mathematics exam, only 80 050 (51%) passed 
representing 17% of the total number of candidates entering the matriculation exams in that 
year. By 1998, although the enrolment figure had increased by more than 120 000 pupils, the 
pass rate had dropped to 42%. As data by racial group is no longer reported, it is not possible to 
gauge the current status quo in 1999. It appears however, that more African pupils are enroling 
in these subjects, but that initially there has been a corresponding drop in the overall pass rate. 

In 2000, out of a total of 741 388 full time and part time students writing matric 4 , 57,9% of all 
the matriculants wrote the mathematics examination (429 263 candidates) 5 . However, only 
45,1% of the candidates passed 6 . The national pass rate for mathematics had dropped in 1996, 
1997 and 1998, but increased in 1999 by 1,3% and in 2000 it increased further by 1,7%. One of 
the reasons for the 3% improved pass rate (1999-2000) is that many weaker students (primarily 
repeaters) who should have written the matriculation examinations were denied access to the 
examinations in 2000 and secondly many students who in the past had taken mathematics on 
the higher grade (a more advanced level of mathematics) had opted to take it on standard 
grade (the most commonly taken and less advanced level) and therefore presumably achieved 
better results at this level. The conclusion that may be drawn though is that although nearly two 
thirds more students enroled for mathematics and wrote the examinations in 2000, only 
approximately 113 000 more students are going through the system successfully compared to 
1993 and most of these at a lower level compared to those in 1993. 

5. Background on TIMSS-R 

5.1 The Third International Mathematics and Science Study 

The Third International Mathematics and Science Study, conducted under the auspices of the 
International Association for the Evaluation of Educational Achievement (IEA) is the single 
largest international comparative study (see Beaton et al, 1996a&b) ever conducted in 
education. In 1995, 41 countries (including South Africa), 15 000 schools and 500 000 students 
participated in over 30 languages. For more details on the South Africa’s participation and 
results in TIMSS in 1995 (see Howie, 1996 and Howie and Hughes, 1997 for details). 

5.2 Design of TIMSS and TIMSS-R 

The lEA’s TIMSS-Repeat was a replication study that followed four years after TIMSS, in 1999 
(see Howie, 2000 for details on South Africa’s 1999 results). One hundred and ninety tousand 
pupils participated in 6 000 schools in 38 countries participated. Tests were conducted in 34 
languages. Several sources of data were collected through mathematics and science tests, 
questionnaires, performance assessment tests and a curriculum analysis project. 



4 A decrease of 4,2% (21 533) in the number of candidates compared to 1999. 

5 An increase of 271 562 students compared to 1993. 

6 This refers to the combined pass rate for higher and standard grade (the two possible options meaning a more advanced or less advanced 
level respectively). 
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Three populations were tested in TIMSS and only one in TIMSS-R (Population 2). These are 
described as populations 1, 2 and 3. Population 1 comprises the students in the pair of 
adjacent grades that contained the most students who were 9-year-olds at the time of testing. 
Population 2 consists of students in the pair of adjacent grades containing the most students 
who were 13-years old at the time of testing. Finally, Population 3 includes students in the last 
year of secondary school, regardless of the type of programme in which they were enrolled 
(Robitaille and Garden, 1996). 

The conceptual framework for TIMSS was derived from previous studies conducted under the 
auspices of the International Association for the Evaluation of Educational Achievement (IEA). 
The IEA studies traditionally have recognised the importance of the curriculum as a variable for 
explaining differences among national school systems and accounting for student outcomes. 
These studies represented an effort to understand education systems and to make valid 
comparisons between them. The curricula and teaching practices of different national systems 
were investigated and compared with the student outcomes. 



5.2 South African sample 

The South African sample in TIMSS-Repeat 1998 of 225 schools (including about 10 000 
pupils). The TIMSS requirements stipulated that a minimum of 150 schools be tested and that a 
minimum of one class (preferably one whole class) per school be tested. The South African 
sample was expanded to 225 to accommodate the inter-provincial analysis required. The 
sample drawn is nationally representative and is stratified by province, school sector and 
medium of instruction. In total South Africa’s sampling frame for TIMSS-R included 7,234 
secondary schools with 968,857 students. Ultimately, more than 9000 students in 200 schools 
were tested however, after data cleaning, 194 schools and 8147 students were included in the 
international dataset. 

5.4 Instruments 

The following instruments were used in TIMSS-R: eight test booklets containing mathematics 
and science achievement tests, pupil questionnaire, mathematics teacher questionnaire, 
science teacher questionnaire, school questionnaire. All of the questionnaires were completed 
as far as possible on the day that pupils were tested. In addition, a National Option was 
included: an English language proficiency test specifically for South African pupils. This 
instrument had previously been validated by the Human Sciences Research Council and 
standardised for Grade 8 Second Language pupils in South African schools. Questions about 
language use were also included in the TIMSS-R pupils' and teacher' questionnaires, to 
ascertain the extent to which the pupils are exposed to English. They included pupils’ home 
language, ethnic group, the language spoken predominantly by the pupils in the mathematics 
class, the language used by the mathematics teacher in class, media languages pupils are 
exposed to and the language of their reading materials. 

6. Design of this research 

The conceptual framework presented in this section (6.1) refers to the larger project mentioned 
in section 1. Nonetheless, it is highly relevant for discussion in this paper as a starting point for 
the analysis of the school-level analysis. As it is essential to understand the conceptual 
underpinnings of the model as a whole the model will be discussed in its entirety, although only 
briefly. 
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6.1 Conceptual framework 

To monitor a dynamic system such as education, it is important to depict this in a way that, 
linkages between components of the system can be ascertained and evaluated. The model 
provided is based on the original framework by Shavelson et al (1987) who did a 
comprehensive review of the literature relating to social indicators and educational research. 
Furthermore, some of the IEA thinking on curriculum is reflected in the model. A number of 
adaptations have been made the original frameworks to better suit the research questions 
posed by this study. 

Figure 1 Factors related to mathematics achievement 

(Howie, 2000b, September 2000, adapted from Shavelson, 1987) 



INPUTS PROCESSES OUTPU 




The model shown in Figure 1 presents the education system in terms of inputs, processes and 
outputs. In the model, the inputs are the policy-related contexts on a national, provincial and 
local level from which the intended curriculum is also designed and developed. The inputs also 
include the antecedents: the economic, physical and human resources supplied to different 
levels of the system; the characteristics of the teachers and the background of the students. 
The inputs into the system affect all the processes of education, which may also be seen as the 
practice in education. Different processes (relating to what is taught and how it is taught) take 
place within the districts, schools, and inside the classrooms in terms of the implemented 
curriculum, teaching (in the meaning of the context and conditions under which teachers work) 
and instruction. The outputs, also seen as the outcomes, eventuate in terms of the 
achievement of learners in specific subjects such as mathematics; participation in class and 
school activities and finally learners’ attitudes towards subjects and schooling and aspirations 
for the future. It is expected that, due to the dynamics of the processes as shown in the model, 
there will also be indirect benefits and outcomes, such as improved learner participation partly 
due to improved curriculum quality. 
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Looking at school-level, the focus of this paper, School quality includes aspects such as 
organisational management and development, governance, financial management, parent and 
community support, human resource management, instructional time, organised curriculum, 
school administration, effective support from district/education system, physical resources, 
school profile and schools’ previous achievement. School quality transcends both the Inputs as 
well as the Processes. This is because school quality comprises elements that are of a macro 
nature (and therefore are an input to the process on school level) as well as a number of factors 
that are typically found on school level and where schools may differ. Teaching requirements 
relates mainly to teaching load, class size, demands on time, teachers perceptions of working 
conditions, autonomy and collegiality. Information about the curriculum quality includes 
management of the curricula, instructional strategies, assessment and selection of curriculum 
materials. Instructional quality on the other hand relates to teaching and learning activities in the 
classroom and includes issues such as instructional resources, instructional policies, processes, 
activities, instructional climate, and teacher and learner interaction. 

Aspects related to and impacting on elements of the system also impact on students’ 
achievement although indirectly. For instance inputs in the education system includes 
contextual factors such as average class size in the district, economic factors such as annual 
per pupil operating expenditure, subsidies to schools, the average parent education attainment 
and annual per pupil capital expenditure. 

The curricula for academic subjects play a central role in an education system. The IEA 
believes that the curricula are key in the evaluation of educational achievement. They 
differentiated between the intended, the implemented and the attained curriculum. In the model 
(Figure 1), central positioning of the intended, implemented and attained curricula and their 
linkage between elements within the model illustrate the key role of the curricula. The intended 
curriculum created through policies on education affects students achievement by outlining what 
the student is expected to learn and the teachers are expected to teach. The intended 
curriculum may occur both on system and school level. The implemented curriculum occurs 
also on two levels, namely at the school and classroom levels. Here the intended curriculum is 
interpreted by the school staff and organised on school and classroom levels. In particular, 
further interpretation of the curricula is made by individual teachers as to what is actually taught. 
The attained curriculum is reflected in the model as outcomes from the education system. 
These include the actual achievement of students in subjects, the level of participation by the 
students in classes and in school activities and finally is also reflected in terms of attitudes 
towards school, their school-work and the subjects in general, their peers and their teachers. 

Other factors are also believed to influence achievement. From the literature, it is clear that 
antecedents such as student background (such as age, gender, home background and socio- 
economic conditions), and teacher characteristics (with similar information) also affect students’ 
achievement. 

The model serves as an important theoretical and conceptual basis for the analysis of the 
TIMSS-R data. As the data was collected on a number of education levels, namely, school, 
classroom and learner level, the model serves as a guide to explore the causal links for the 
learners’ achievement. 

6.2 Research questions 

The following research questions will be addressed in this paper: 

1. How does the performance of the South African pupils in mathematics compare with pupils 
from other countries? 
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The mathematics results of all the countries participating in TIMSS 1999 will be compared 
with South Africa, including the subscales. 

2. How do pupils from different language groups perform in the mathematics test? 

In 1995, about 80% of the South African pupils wrote the test in a language that was not their 
main language and there were indications that this affected their performance negatively. In 
1999, 74% of the pupils wrote the test in a languages that was not their main language. 

3. How did pupils perform in the English Language proficiency test and how is this related to 

achievement in mathematics? 

The extent of the pupils’ aptitude in English is analysed and described. Every South African 
pupil completed a writing skills and language usage test which was a standardised test for 
second language speakers designed and developed by the Human Sciences Research Council 
in South Africa. The performance of the pupils in this test indicates a measure of language 
proficiency in English. In particular, the performance of pupils whose main language is not 
English but receive their instruction at school in English will be analysed. As language in South 
Africa is traditionally related to other important background variables due to the nature of the 
country’s political past, other background variables will be explored and included in a model on 
pupil level. 

4. what background variables on school-level are related to South African pupils achievement in 

mathematics? 

Factors correlating with achievement will be identified and included into a PLS model for further 
analysis. Thereafter the most significant relationships willb e identified amongst the school-level 
variables and achievement. 

6.3 Research methods 

In order to explore the relationship and the effects of the different levels (and of the different 
variables) on students’ achievement in mathematics, the main research project (on all three 
levels- school, class and student) is divided into two parts. 

The first describes the South African students’ performance in mathematics and language and 
will also provide descriptive information regarding the background characteristics of the 
students, their mathematics teachers and the schools that they attended. The second part is 
the exploratory phase of the study and the secondary analysis of the TIMSS-R data related to 
mathematics achievement. The data are explored to investigate the reasons for the students’ 
performance described in phase 1 and to explore the inter-relationships of achievement and the 
background variables revealed by students, teachers and the school principal. In particular, the 
exploratory part of the study proposes to determine the factors that influence mathematics 
achievement and performance of South African students and to ascertain the influence of South 
African second language students’ language and communication skills of in particular on their 
achievement in mathematics. This exploration is intended to reveal a number of relationships 
between variables level that are open to manipulation on a student level, class and school level. 
Whilst it is recognised that some exogenous preconditions, which are factors external to the 
school (Brummelhuis, 1995: 15) (for example socio-economic variables) cannot be manipulated 
by the school, it is expected that there are a number of endogenous factors (on system-level 
and within the school or class) that may be manipulated. The intention of the study is to reveal 
those exogenous and endogenous factors through this exploration. 

In this paper the descriptive data with regard to mathematics and English are presented as well 
as the data at school level from the school questionnaire and the results of the first exploratory 
analysis on school-level are discussed. 
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6.4 Data Analysis 

The data exploration aiming at identifying factors that influence achievement in mathematics in 
South Africa require scale and path analysis. The first step in the analysis plan was to produce 
univariates of the mathematics and English achievement data as well as all the possible school- 
level factors linked to the research questions and the data were explored to make constructs 
(such as mathematics resources in the school) and thereafter to make a correlational matrix. In 
the descriptive phase bivariate and multivariate analyses were also done. These preliminary 
results are described in section 7.1. Principal components analysis and reliability analysis were 
carried out on sets of items referring to one factor or construct. Sets of items with a reliability 
coefficient Cronbach a of at least .50 were selected as composite variables. Thereafter, the 
correlational matrix was important to identify possible variables linked to achievement, to build 
constructs and to prepare a basic model for further analysis in second part of the research (see 
6.3). 

For the secondary analysis on TIMSS-R data, the PLSpath approach was applied. ‘PLSpath’ 
stands for ‘Partial Least Squares path analysis’ technique (Sellin, 1990, 1992). PLSPath version 
3.0 was used in this research. This computer programme was developed by Sellin (1990) and 
is based on the Partial Least Square (PLS) procedure. PLS was introduced by Wold (1982) as 
a method for exploring relationships of independent variables by estimating path models with 
latent constructs measured by multiple indicators. PLS is a flexible and extremely powerful 
technique for the examination of path models with latent constructs measured by multiple 
indicators (Sellin, 1995:266). This is due to the fact that it can handle big datasets, it is 
technically easy to use, is very quick in computing the outcomes and finally does not require 
rigorous distribution assumptions. Since its development, many researchers have used it to 
analyse large datasets (Bos, (2000), Mahondas, (1999), Afrassa, (1998), Lietz, (1995), Sellin 
and Keeves, (1994) and Keeves, 1986 amongst others). Researchers such as Mahondas claim 
that PLS is very flexible in the initial analysis and that it gives satisfactory results. As the path 
model was developed post hoc (decisions concerning instruments and associated variables 
were made before the model was developed), the nature of the analysis is seen as more 
exploratory than confirmatory. The PLS technique was developed especially for research 
situations that require a great deal of exploratory analyses. In contrast, other approaches like 
LISREL and AMOS, were designed primarily for situations that require confirmatory tests of 
theoretically well-established path models. For more details of the PLSpath technique, 
publications of Sellin and Keeves (1994), Sellin (1990, 1992), and Wold (1982) are 
recommended. 

Given that there are a number of variables influencing student’s achievement and that some of 
these are intricately inter-related, Partial Least Squared (PLS) analysis was used initially to 
explore firstly the school-level factors. In the near future, it will be used to analyse the student- 
level and classroom-level factors that influence students’ achievement in mathematics. 

In the larger project (and not included in this paper), due to the fact that data were collected on 
three levels - student-level, class-level and school level, the Hierarchical Linear Modelling 
(HLM) will be used (this will be decided after the results of PLS are known) to distinguish 
between the variance in mathematics achievement uniquely explained by student-level factors 
as opposed to the variance uniquely explained by the classroom and school-level factors. As 
only one class per school was sampled, only two levels will be considered for the Hierarchical 
Linear modelling. As this stage has not been reached yet, only the preliminary results from the 
school questionnaire data related to the school-level model are included in this paper. 
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7. Results 

7.1 Performance of South African pupils in mathematics compared to pupils in other 
countries 

7.2 

TIMSS-R (as did TIMSS) used item response theory methods to summarise the achievement 
results. A scale of 800 points was used with a mean of 500 and a standard deviation of 100 
points. Figure 2 illustrates pupils’ achievement in mathematics from all 38 participating 
countries, giving the scale score out of 800 points. Countries are given in descending order of 
mean (average) achievement. The average scale score (with the standard errors), the years of 
formal schooling, and average age of the pupils are also presented. 

<lnsert Figure 2 Distrib u tion of Mathe mati cs Ac hievement about here> 

South African pupils performed poorly when compared to all of the participating countries. The 
mean score of 275 (standard error [SE] 6.8) is well below the international mean of 487 (SE 
0.7). This result is significantly below the mean scores of all other participating countries, 
including the two other African countries of Morocco and Tunisia as well as that of other 
developing or newly developed countries such as Malaysia, the Philippines, Indonesia and 
Chile. The achievement in mathematics was dominated once again by the Asian countries of 
Singapore (at the top with 604 scale points), Korea, Chinese Taipei, Hong Kong and Japan who 
achieved the highest scores. 

The percentiles (see Figure 2) provide a perspective on the size of the differences between the 
countries. This is useful to review the performance of South African pupils on the international 
test compared to other countries. The top South African pupils (at the 95 th percentile) were 
comparable to the below-average pupils from Singapore, indicating the vast difference between 
the two countries. This means that only the most proficient pupils in South Africa - and 
incidentally the same holds for Chile, Morocco and the Philippines - approached the level of the 
average pupils from Singapore. The South African pupils scoring around the country’s mean fell 
below the least proficient pupils from almost all other countries with the exception of Morocco, 
the Philippines, Chile and Indonesia. 

Virtually all the pupils participating in TIMSS-R had had eight years of formal schooling. There 
were a few exceptions. For example, some of the Australian pupils had received nine years of 
formal education, as had all the English pupils; pupils from Finland and the Philippines had 
received only seven years of schooling. 

South African pupils, on average, were the oldest pupils in TIMSS-R as they were 15.5 years of 
age. This was significantly above the international average age of 14.4 years. The only other 
country where the average age was above 15 years was Lithuania (15.2). 



7.2 Performance of South African pupils not speaking the language of the test at home 

There was a substantial difference between the pupils who came from homes where the 
language of the test was always or almost always used (see Table 1). There was nearly 100 
scale points difference between those who always/almost always spoke English or Afrikaans at 
home and those who sometimes did and an even greater difference between those never did. 
Similarly, there were large differences between the three groups for each of the sub-scales with 
the smallest difference in scores being observed in geometry. Overall and for each of the sub- 
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scales, pupils who spoke English or Afrikaans at home performed better than those writing the 
test in an additional language. 

Table 1 Mean mathematics overall scores and sub-scale scores of pupils who always, sometimes 
or never spoke the language of the test at home 





Always / 
almost always 


Sometimes 


Never 


Mathematics score 


360 


263 


231 


Algebra 


382 


277 


249 


Data Representation, analysis & probability 


406 


352 


327 


Fractions & number sense 


366 


294 


249 


Geometry 


374 


326 


321 


Measurement 


388 


317 


301 



The low scores of the second language learners are most obvious when the pupils’ scores on 
the test written in Afrikaans are compared to those who wrote in English. In South Africa, most 
of the pupils who speak one of the African languages at home would most likely attend schools 
where English is the medium of instruction and therefore they are learning in a “foreign” 
language. In contrast, pupils who attend schools where Afrikaans is the medium of instruction 
are most likely native Afrikaans speakers and would be white Afrikaners or Afrikaans-speakers 
of mixed descent (the coloured people). As can be seen below, most of the pupils writing the 
test in Afrikaans performed well above those writing the test in English. About 20% of the 
learners wrote in Afrikaans and once again the topic where the smallest difference in scores 
was found was in geometry. 

Table 2 Overall mathematics scores and sub-scale scores of pupils writing the test in English 
and Afrikaans 





English 


Afrikaans 


Mathematics score 


260 


376 


Algebra 


275 


398 


Data Representation, analysis & probability 


347 


421 


Fractions & number sense 


285 


380 


Geometry 


327 


385 


Measurement 


318 


397 



7.3 Performance of South African pupils on the English language proficiency test 

In general, the performance of the South African pupils on the English test was also poor. The 
mean score for all the pupils was 16 out of 40. The test however is considered reliable as the 
reliability coefficient was 0.81. What is immediately obvious is that the provinces whose pupils 
performed relatively better in the TIMSS-R mathematics test, namely, Western Cape, Gauteng 
and Northern Cape, also performed better in the English test. In the Western Cape and 
Gauteng, these areas are largely urban and therefore one might expect the pupils to be more 
exposed to English than those in some of the provinces comprising vast rural areas. However, 
one interesting aspect is that pupils in the Northern Cape, most of whom were primarily 
Afrikaans-speaking performed better than those in other provinces. The explanation for this 
relatively better performance may well lie at school level. The pupils from the Northern Province 
obtained the lowest overall mean scores in both mathematics and English. This largely rural 
province also has one of the highest unemployment rates, a very low socio-economic basis for 
most inhabitants and relatively few resources for one of the largest school-going populations in 
the country. 
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Table 3. South African pupils scores for the English test and the mean overall scores for the 
TIMSS 1999 mathematics test. 





Province 


English 

Mean score 


Maths 

Mean score 


SE 


1 


Eastern Cape 


15 


256 


11.8 


2 


Free State 


16 


266 


20.3 


3 


Gauteng 


21 


326 


22.7 


4 


KwaZulu Natal 


16 


271 


17.3 


5 


Mpumalanga 


15 


247 


15.2 


7 


Northern Cape 


18 


315 


13.6 


6 


North West 


16 


267 


12 


8 


Northern Province 


14 


229 


4.7 


9 


Western Cape 


21 


376 


20.7 



A Pearson’s correlation coefficient of 0.65 was found when testing the relationship between the 
mathematics and English test scores indicating a strong relationship between the two. Those 
pupils who achieved better scores in mathematics also tended to do so in English. Perhaps this 
link between the performance on the English test and that on the mathematics tests is explained 
by the fact that language plays an important role in the formulation and expression of concepts 
in mathematics. For instance, language is often meaningless to pupils when using it as their 
second language and this results in a lack of confidence and unwillingness by pupils in using 
the mathematics language (Colyn and Lebethe, 1998: 23). This is also because of the use of 
familiar words in unfamiliar contexts; the ambiguity of the language of mathematics; and the fact 
that learners informal language is much more developed and complex than their mathematical 
language which is further complicated by their poor grasp of the language of learning. 
Therefore, this has serious implications generally for the achievement of second-language 
speakers and in particular, for their performance in mathematics. 

This was confirmed by the fact that there was a strong correlation between those pupils 
speaking the language of the test at home and their performance on the Mathematics (0.43) 
and English (0.45) tests. Those pupils who always or almost always spoke the language of the 
test at home were therefore more likely to perform better in mathematics and English. 
Conversely, the less that the pupils spoke the language of the test the worse their test scores in 
both mathematics and English. In particular, this indicates that the second language pupils, 
primarily many African pupils, performed poorly in both mathematics and English. 



7.4 Contextual factors on school-level 

As mentioned previously, the study sought explanations for pupils’ achievement in mathematics 
and in this paper the first level investigated was the school-level. In this section, the preparatory 
steps as well as the results of the school level analysis are given. 

7.4.1 Selection of School Quality items for analysis 

A review of the school questionnaire revealed that almost all the items could be categorised 
under the component School Quality in the conceptual framework described in section 6. After 
examining the univariates combined with the literature review, the following factors were 
identified for further analysis: 
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Human resources 
Selection of students 
Learning environment 

Principals’ activities 

Parental involvement 
First language 

School enrolment 
Repeaters 
Class size 

Community 

Retention of the 
teaching staff 
Limitations 

Autonomy of the staff in 
the school 



the number of teachers in the school, 

admission procedures followed by the school to admit students, 

percentage of students absent on an given day, the frequency of negative behaviour 

and the principal’s perception of the gravity of this behaviour, 

these included activities related to instructional leadership, communication, 

administration and communication, 

schools’ expectations of what parents should do at school 

the number of students whose home language was the same as the medium of 

instruction in the school, 

the number of students enroled at the school, 

the percentage of students repeating grade 8, 

the average number of students across the grade 8 classes), Grade size (the number 
of students in grade 8, 

the location of the school: in isolated area or village, rural town, outskirts of a city and 
city centre, 

the percentage of teachers who have been at the school for longer than 5 years, 

shortages of general facilities and learning equipment, shortages of maths-related 
facilities and learning equipment, 

responsibility for taking decisions - outside school, school governing board, school, 
heads of departments and teachers. The extent to which the staff at schools playing 
a role with regard to influencing the curriculum and specifically the role of teacher 
unions in influencing the curriculum implemented at the school. 



The following section describes the univariates from which the factors described above were 
derived. The scale analysis for the factors that are composites is discussed separately in 7.5.1 . 

7.4.2 Profile of the schools in TIMSS-R 

Human Resources 

The number of teachers varied markedly across schools. On average schools had 21 full-time 
teachers with the smallest school having one teacher and the largest teaching staff comprising 
100 teachers. Few schools had part-time schools, the mean being 0.8 part-time teachers per 
school and the highest number of part-time teachers found was 25. Principals reported that on 
average 68% of their teachers had been at the school for 5 years or longer. 

School enrolment 

The school enrolment was vastly different across the sample with the smallest enrolment being 
44 students and the largest being 1957, typically representing a farm school in a rural area in 
the former and an urban school, the latter. On average schools had 854 students enroled. 

Class size in grade 8 

In South Africa, class sizes are generally larger than in most developed countries. In this 
sample, the average grade 8 classroom had 46 students in it, although the largest class found 
had 95 students. 

Learning environment 

The ethos in the South African schools deteriorated over the past 25 years, although there is 
some evidence that suggests that there is more stability in the last couple of years. However, it 
was no surprise to find that principals reported a relatively high incidence of negative behaviour 
in many schools and that in many cases they perceived this behaviour to be of a serious nature. 
These reports are summarised in table 6.1. 
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Table 4. Incidents of negative behaviour reported by the school principals 



Incidents of negative behaviour 


Frequency of behaviour 

(n = 188) 




Never or 


Weekly or 


Daily 




rarely 


Monthly 




Theft 


68 


27 


5 


Intimidation of or verbal abuse of other students 


60 


29 


11 


Physical Injury to students 


87 


10 


3 


Intimidation or verbal abuse of teachers 


90 


9 


1 


Physical injury to teachers 


98 


1 


1 


Illegal drug use or possession 


82 


9 


9 


Weapon use or possession 


86 


12 


2 


Inappropriate behaviour 


92 


7 


1 



In several cases, principals in South Africa reported a higher incidence of certain behaviours 
than was the case internationally, particularly with regard to the daily occurrence of theft, 
intimidation of students, alcohol abuse and drug abuse or possession. Evident is that 11% of 
the principals are dealing with intimidation and verbal abuse of other students by grade 8 
students on a daily basis and nine percent are faced with problems of illegal drug use or 
possession by grade 8 students. Theft is reported as frequently occurring in a third of the 
schools as often as monthly, weekly or daily. Most shocking are reports of actual physical injury 
to teachers even though these are reported in only two percent of the schools, however more 
common is the intimidation or verbal abuse of teachers by grade 8 students reported by 10% of 
the principals. 

Principals activities 

Principals were asked to estimate how much time was taken up by activities related to 
instructional leadership (discussing educational objectives with teachers, initiating curriculum 
revision and planning, training teachers and professional development activities); 
communication (talking with parents, counselling or disciplining students and responding to 
requests from district, provincial and national education); administration (hiring teachers, 
representing the school in the community, representing the school at official meetings and 
internal administration tasks) and teaching (including preparation) or giving demonstration 
lessons. 

Table 5 Hours as reported by principals spent by on different activities at school 



Instructional 

leadership 


Communication 


Administration 


Teaching 


Other 

activities not specified 


19.5 hours 


35 hours 


18 hours 


27 hours 


9 hours 



Most of the principals’ time they reported was taken up by communication activities (35 hours). 
Principals also spent a considerable amount of time teaching (27 hours). In comparison, little 
time was spent on instructional leadership and administration. 

Principals' expectations of Parental involvement 

On the whole, principals reported that they expected parents to be involved in activities related 
to the school and related to their children’s learning. Principals were asked to indicate what 
their schools expected from the parents. Most principals indicated that schools expected 
parents, to volunteer for school projects and programmes, to ensure that their children complete 
their homework, assist teachers with their trips, prepare food for children to take to school, serve 
on committees that select staff for the school and on those that review school finances. Most 
schools do not expect parents to notify the school about problems their child was having at 
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home or with classmates, to raise funds for the school or to patrol the grounds of the school to 
monitor student behaviour. 

Community 

Half of the schools in the sample are based in rural areas with 3% of them being regarded as 
very isolated and 47% in rural towns or villages. Of the urban schools, 20% are on the outskirts 
of the cities and 30% in the city centres. A clear distinction was found in the results of the 
schools in these areas. The mean score of the rural schools was the lower than the urban 
schools, 225 and 227 compared to 287 and 328 points on a scale of 800 respectively). 

Limitations 

Principals were asked to report on the factors that they perceive affect instructional capacity of 
the school. Although they rated them affecting the capacity none (not at all), a little, some and 
a lot, Table 6 reports only none and a lot as well as the mean scores for each group. 

Table 6 The extent to which the shortage of resources affects the instructional capacity 
of the school 



Limitation 


Extent of the limitation affecting the instructional 
capacity of the school 
% of principals 




Not at all 


Mean 


A lot 


Mean 




% 


score 


% 


score 


i General shortages of general resources 










Instructional materials (e.g. textbooks) 


11 


303 


46 


259 


Budget for supplies (e.g. paper, pencil) 


16 


295 


40 


261 


School buildings and grounds 


21 


289 


40 


255 


Heating/cooling & lighting systems 


20 


284 


42 


246 


Instructional space (e.g. classrooms) 


27 


277 


32 


247 


Shortage of Mathematics related resources 








i 


Computers for mathematics instruction 


20 ~ 


253 


69 


260 


Calculators for mathematics instruction 


20 


296 


47 


251 


Library materials relevant to mathematics 


14 


239 


59 


254 


Audio-visual resources for mathematics instruction 


13 


247 


64 


250 


Teachers qualified to teach mathematics 


23 


290 


27 


252 



Regarding general resources, in all cases there is a clear difference between these two groups 
in achievement between schools reporting limitations and their achievement. Those that report 
that their schools are not affected by general shortages have a correspondingly higher 
achievement score in mathematics. However, the relationship is not so clear with regard to 
mathematics-related resources. This is the case for computers, library materials and audio- 
visual resources where the mean score for those reporting these not affecting their capacity was 
lower than reporting it affecting their capacity a lot. This seeming incongruous outcome is 
probably due to the fact that principals (from higher performing schools) reporting that they feel 
it affects their schools’ capacity as well as the principals’ (from lower performing schools) who 
do not perceive the value or linkage between those resources and their schools’ capacity to 
deliver instruction in mathematics, answered the same answer category. Teachers of 
mathematics who actually experience the limitations first-hand would probably answer the same 
questions quite differently. The more obvious linkages to a school’s capacity to provide 
instruction in mathematics, namely calculators and teachers of mathematics follow the same 
pattern as the reports under the general resources, that is that those without resources attain 
lower means scores. 
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Autonomy of the staff in the school 
Under this category, three items were investigated, namely who is primarily responsible for a 
number of activities in the school, secondly, the stakeholders within the schools’ influence in 
determining the implemented curriculum at schools and finally, the extent of the teacher unions’ 
influence in determining what is included in the implemented curriculum. 

As can be seen from Table 7, most of the responsibilities for the executive functioning of the 
school lie with the principal. The heads of departments have a role with regard to the placing 
students in classes, determining the textbooks to be used and establishing policies. What is 
striking is the apparent absence of teachers, with regard to having responsibilities in the 
functioning on school level in the majority of schools. The growing number of schools installing 
school governing bodies is becoming more evident with the majority of schools reporting that 
these boards are responsible for hiring teachers, establishing disciplinary policies and 
formulating the school budget. 

Table 7 Stakeholder responsibility at schools 



Activities 



Not a school School 
responsibility governing 
board 



Principal 



Department 

head 



teachers 



Hiring teachers 


X 






Establishing disciplinary policies 
Establishing student grading policies 


X 


X 




Formulating the school budget 
Purchasing supplies 


X 


X 




Placing students in classes 
Assigning teachers to classes 




X 


X 


Determining which textbooks are used 






X 


Establishing homework policies 

Determining teacher salaries X 

Establishing community relationships 




X 


X 


Communicating with students families 




X 




Determining course content 




X 




Deciding which courses are offered 




X 





Influencing the implemented curriculum at schools 

It appears that in most schools, individual teachers also have little influence with regard to the 
curriculum implemented at their school on school-level. Most of the influence seems to come 
from the provincial education department, although the principals report that they have some 
influence as do the department heads and the teachers collectively. As this interpretation has 
been made on data derived from the school principal questionnaire, the teacher questionnaire 
data will also be explored carefully to ascertain to what extent this is true on class-level as well. 

7.5 Factors on school-level influencing achievement in mathematics 

In order to prepare the data for model building, a number of preparatory steps need to be taken 
(Bos, 2000). First the descriptives and frequencies of all possible variables have to be 
analysed. As PLSPATH does not accept variables with missing values, these have to be 
replaced. As aggregated student level data was used (with regard to home language of 
students and mathematics scores) and then merged with the school-level data, a number of 
cases (n=10) were dropped where student-level data was not available. The final sample size 
for South Africa was 189 schools. Once the missing values had been replaced, the correlation 
matrix was prepared and the factor analysis and reliability analysis were conducted for each 
variable in the model. In this next section, the reliability analysis (7.5.1) is described followed by 
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a summary of the results of the correlational analysis (7.5.2), and thereafter a description of 
developing and exploring the model and finally the results of the PLS analysis in 7.5.3 . 

7.5.1 Reliability Analysis 

A number of these factors were composites developed through a process of factor analysis 
(including principal component analysis) and reliability analysis. The reliability coefficients are 
given in Table 8 for each of the constructs described previously. If the Cronbach alpha 
coefficient was higher than .50, it was considered suitable for inclusion in the further analysis 
and is given in Table 8 

Table 8 Results of reliability analysis 



Factor 


Individual variables 


Cronbach alpha 


Selection 


Admission procedures (13 items) 


.75 


Learning environment 


Frequency of negative behaviour (9 items) 


.87 


Principal activities 


Hours spent by principal on selected activities at school (14 
items) 


.51 


Parental involvement 


Schools' expectations of the extent of parents involvement in 
schools' activities (6 items) 


.74 


Limitations 


Shortages of general resources (5 items) 


.75 




Shortages of maths related resources (6 items) 


.87 


Autonomy 


Responsibility for Decision-making in school 
(14 items) 


.65 




Stakeholders influence of implemented curriculum 
(4 items) 


.85 



7.5.2 Correlational Analysis 

After the reliability of all the constructs was tested, a correlation matrix was analysed. Each of 
the constructs and the other previously identified factors that are single variables were 
examined in relation to the mean mathematics scores. The results of this analysis are found in 
Table 9. Of the 17 individual variables (clustered into 13 factors), 10 of these were found with a 
coefficient of higher than 0.15, which was taken as the cut off point for inclusion in further 
analysis and as indications for direct paths to mathematics achievement. 



Table 9 Correlation of background factors with mathematics score 



Factor 


Individual variables 


Pcorr. Math 
score 


Human resources 


Number of teachers in school 


.39** 


Selection 


Admission procedures (13 items) 


-.09 


Learning environment 


Absenteeism 


-.20** 


Frequency of negative behaviour (9 items) 


.00 


Principal activities 


Hours spent by principal on selected activities at school (14 items) 


++ 


Parental involvement 


Schools’ expectations of the extent of parents involvement in schools’ activities (6 items) 


.13 


First language 


Number of students in the tested class whose first language is the same as the medium of 
instruction 


.58 


School enrolment 


Number of students enrolled in school 


.28** 


Repeaters 


Number of students repeating grade 8 


-.20** 


Class size 


Average number of students in grade 8 classes 


-.22** 


Community 


Urban-rural community where school is located 


.36** 


Retention of 
teaching staff 


% of teaching staff who have been in the school longer than 5 years 


-.04 


Limitations 


Shortages of general resources (5 items) 


-.18* 




Shortages of maths related resources (6 items) 


-.19** 


Autonomy 


Responsibility for Decision-making in school (14 items) 
Stakeholders influence of implemented curriculum (4 items) 


-.10 




Influence of teacher union on curriculum 


-.05 

-.22** 



* significant at the 0.05 level, ** highly significant at the 0.01 level ++ Not calculated due to large % of missing data (more than 15%) 
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No correlation was found between selection of students into schools, the reported frequency of 
negative behaviour (under learning environment), retention of teaching staff and stakeholders’ 
influencing the curriculum (under autonomy). Furthermore, responsibility for decision-making in 
the school (under autonomy) and parental involvement had showed low correlations (-.10 and 
.13 only). The most highly correlated items with maths achievement were first language 
speaking students and the community where the school is located. The more first language 
speakers in the school, the higher the achievement score was. As this variable is also 
associated with socio-economic group status in South Africa this finding is to be expected. 
Students speaking English or Afrikaans are generally expected to come from more advantaged 
backgrounds than the majority of students speaking African languages at home. In addition to 
that, schools attended by the majority of first language speakers are considered privileged in 
their resources and generally have better facilities, equipment and more highly qualified 
teachers. It is important to note here that the exception to this would be the schools, which 
previously contained only Indian (typically having English as their home language in addition to 
Indian languages) and coloured (of mixed race) students (having either, English and Afrikaans 
as a home language). These schools would also have been disadvantaged under the old 
dispensation having poorer resources than former White schools, but generally slightly better 
than those of African schools. 

The correlation between community and maths achievement indicates that the schools in the 
more urban areas produced better results than in rural areas. Again, this is perhaps not 
surprising as urban schools, on the whole, have better resources and attract students from 
more privileged backgrounds than rural students. 

7.5.3 Developing a school model for mathematics 

It is recommended by Falk (1987) that drawing a path diagram is helpful when starting to build a 
model using PLS. Therefore, prior to the analysis of the data, a hypothesised model, was 
compiled. This was done conceptually on the basis of what school-level factors could be 
expected (from the literature and knowledge of the context) to influence students’ achievement 
in mathematics. It is hypothesised that a number of factors influence achievement directly (such 
as the number of repeaters in grade 8, the class size, parental involvement, the learning 
environment amongst others) whilst others also influence the achievement indirectly (such as 
school enrolment). Once the correlational analysis was completed, the factors were selected for 
inclusion into an initial model (a reduced hypothesised model). This initial model was tested 
using partial least square analysis. 

7.5.4 Exploring the model using Partial Least Square Analysis 

The first stage in model building is to draw the path diagram and thereafter the model is 
systematically trimmed. This involves eliminating the manifest variables and the latent 
variables, which do not have significant paths in the model (Beta coefficient less than .15). 
There are two parts to the model - the outer and inner models. The outer model specifies the 
relationships between the latent variable and the manifest variables, which either form or reflect 
the latent variable. The inner model indicates the strength of relationships between the latent 
variables. Five criteria are used for trimming the model and these are the weight, loading, 
communality, redundancy and tolerance (see Bos, 2000 for details). 
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Ultimately, at this stage of the school-level model, only unities were entered into the outer 
model, meaning that a latent variable was reflected by a single manifest variable. Therefore the 
results given here only relate to the inner model, which specifies the relationship between the 
latent variables. 

Table: 10 Inner model results of PLS 



Factor 


Beta 

>.15 


Correlation 


R-squared 

>.10 


Repeater 






.69 1 


Teachrat 


~~ .83 


;;; .83 “I 777 




School 






7 21 1 


Communit 


.45 


'.45 




, r Class 






.12 1 


Communit 


-.19 


-.21 




Resource 


.16 


.20 




Firslang 


-.18 


-.22 




School 


.16 






f Math 







.62 1 


Communi 


.19 


.41 




Union 


-.17 


-.25 




Firslang 


.67 


.74 





Table 10 shows that 62% of the variance in the student's achievement in mathematics (MATH) 
can be explained by the variables Community, Union and First language. The results indicate 
that the students from schools in urban communities and students from schools where the 
students speak the medium of instruction at home are highly likely to do better in mathematics. 
Conversely, students in schools where teacher unions’ are influential in determining what is 
taught are likely to perform worse in mathematics than students in schools where this is not the 
case. Although often quoted in the literature as influencing achievement, class size as seen in 
this model (Figure 3) has a very weak relationship (.08) with the mathematics score and 
therefore was not included in the table above, but was included for illustrative purposes below. 
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Figure 3 Final model of school-level factors only 




2 

P 

S- 



As can be seen in the model, the number of repeaters at the school is greatly influenced by the 
teacher:student ratio with 69% of the variance being explained by this variable alone. 
Repeaters are highly likely to come from schools where the teacher: student ratio is high and 
therefore teachers are dealing with larger numbers. The school size is influenced by the 
location it is in and larger schools are found in more urban settings. Class size is influenced by 
a number of factors. Those found include the location/community, the resources in the school, 
whether there are first language speakers and the size of the school. Only one variable (not 
given in the table) contributed indirectly to class size and that was Community (.07) by way of its 
effect on school size. The variance explained (seen by the R-squared coefficient in the table) is 
low and CLASS (.12) only just meets the criteria of .10 recommended by Sellin (1995) for 
smaller samples (such as 189). The larger classes are more likely to be found in rural 
communities, where there are greater shortages of general resources, where most of the 
students speak African languages at home and where the schools have high enrolment 
numbers. 



CONCLUSION 

When South Africa participated in TIMSS in 1995, only student-level could be analysed and 
therefore no profile of the schools could be generated within that study. Therefore the 1999 data 
permit researchers working with this data the opportunity for the first time to link different levels 
of data within a school together. At this stage of the research, the initial analysis of the English 
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language data has been completed. However, only the school-level data (and not the 
classroom and pupil level data) was explored within certain constraints. One of these was the 
limitation of data available within the school questionnaire related to school quality. 
Nonetheless, some important aspects of school quality related to school leadership, parent 
involvement, school profile, physical resources, human resources, autonomy, learning 
environment and school administration were explored. Additionally, two important antecedents 
related to the type of community and the home language of the student were included in the 
model. 

Language problems are dominant in SA currently as policies relating to language are under 
development. The fact that there are 11 dominant recognised languages in the country 
presents certain logistical challenges for the education system (amongst others). Presently, 
English and Afrikaans are still considered the medium of instruction in the majority of schools. 
However, a key problem is that in most of the schools the language of instruction and the 
mother tongue of the teachers and/or the students are different. The result of this is that 
students' achievement in mathematics (and possibly other subjects) is negatively affected, as 
can be seen from the high correlations between language and mathematics achievements. 
From this study it would appear as pupils performing better on the English language test also 
performed better on the mathematics test. In addition the results on provincial level as well as in 
the school-level model indicate that the mathematics results were better where there was a 
larger number of first language speakers (both English and Afrikaans) in the class. It should 
also be noted however that language is a confounding variable as it is also closely related to 
socio-economic status and schools with predominantly better resources. This will be further 
investigated in the larger research project. 

Although, this work is currently in progress, it is however clear that there are only a few 
variables on school-level identified as affecting mathematics achievement. These are largely 
factors beyond the control of the school (namely the location of the school and the home 
language of students) and but nonetheless need to be considered by education planners and 
policy makers. Furthermore, identifying these factors helps to explain the overall results and to 
alert those in authority as to the effect of these variables on students’ achievement in 
mathematics. The influence of the location of the school in rural or urban areas on 
mathematics achievement is not surprising given the under-development in rural areas in South 
Africa. However, as 50% of South Africa’s population live in rural areas, the fact that students 
attending school in rural areas perform worse in mathematics than those attending schools in 
urban areas should be of serious concern to the education and other authorities and policy- 
makers. 

A third factor identified as influencing achievement was that of the extent of the teacher union’s 
influence on the curriculum, which was negatively related to achievement. Although the data 
suggest this relationship, caution needs to be taken with regard to the interpretation of this 
outcome. This is primarily due to the conclusion being drawn based on a single item in only the 
school questionnaire with little further interpretation possible from the original question asked. 

Further investigation into school-level effects will continue as the teacher-level data (from the 
mathematics teacher questionnaire) is combined with the school-level data (from the school 
principal’s questionnaire) to produce a single school and teacher level model. This is due to the 
fact that only one class per school was selected for testing as well as the fact that PLS is a uni- 
dimensional model. Additional variables from the maths teacher questionnaire that will be 
added to and tested include: Classroom resources, teachers’ background, teachers’ confidence, 
activities conducted by the teacher in the class, teachers’ beliefs, maths topics coverage, 
homework, teaching style, classroom environment, medium of instruction in school, teachers’ 
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attitudes to teaching, time on task. Certain aggregated student-level data will also be 
included in the model such as: racial group, age, gender and socio-economic status. 
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